Image processing apparatus, image processing method, and storage medium

ABSTRACT

An image processing apparatus is capable of appropriately extracting a frame of an output target from a moving image. The image processing apparatus includes an analysis unit configured to analyze a plurality of analysis regions in each of a plurality of frames included in the moving image, an extraction unit configured to extract the frame of the output target from among the plurality of frames by comparing analysis results of the plurality of analysis regions in each of the plurality of frames analyzed by the analysis unit for each analysis regions corresponding to each other between the plurality of frames, and an output unit configured to output the frame of the output target extracted by the extraction unit.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image processing apparatus forextracting a frame of an output target from a plurality of framesconstituting a moving image, an image processing method, and a storagemedium.

Description of the Related Art

Conventional apparatuses including a moving-image replay functioninclude a function for extracting, from a moving image, a still imageconstituting the moving image. Using a function of this type, when auser specifies a frame of an extraction target from a plurality offrames included in the moving image, an image of the specified frame istransmitted to a display device such as a liquid crystal monitor to bedisplayed thereon. In addition, the frame extracted from the movingimage can be outputted to a print device to print the imagecorresponding to the frame on a print sheet.

Of the still images included in the moving image, the still imagephotographed when a hand of a photographer or an object moves may have ashake, and thus such an image may not be preferable as an output target.Japanese Patent Application Laid-Open No. 4301236 discusses a techniquein which moving image data is analyzed to acquire an amount of shake ofeach frame included in the moving image, a reference frame is determinedbased on the acquired amount of the shake, and then peripheral frameswithin a predetermined range of the reference frame are output.

As a method for determining the amount of the shake of the image, theabove-described conventional technique discusses a method fordetermining the amount of the shake based on an amount of movement,between the frames, of a feature point included in the frame.

However, even though the feature point included in the frame movesbetween the frames, it cannot be specified whether the feature point ismoved due to the shake of a photographing apparatus or a movement of theobject. Thus, even if the amount of the movement of the feature pointbetween the frames is small and thus the amount of the shake of thecharacteristic point is small, the moving image may be photographedaccording to the movement of the object corresponding to the featurepoint. Further, at this point, even if the amount of the shake of thefeature point is small, the amount of the shake of other region than thefeature point in the frame may be large, and thus the frame may not bepreferable as the output target.

SUMMARY OF THE INVENTION

The present invention is directed to providing an image processingapparatus being capable of appropriately extracting a frame of an outputtarget from a moving image, an image processing method, and a storagemedium.

According to an aspect of the present invention, an image processingapparatus including an analysis unit configured to analyze a pluralityof analysis regions in each of a plurality of frames included in themoving image, an extraction unit configured to extract the frame to beoutput from among the plurality of frames by comparing analysis resultsof the plurality of analysis regions in each of the plurality of framesanalyzed by the analysis unit for each analysis regions corresponding toeach other between the plurality of frames, and an output unitconfigured to output the frame of the output target extracted by theextraction unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating an image processing apparatus.

FIG. 2 is a flowchart illustrating frame extracting processing.

FIG. 3 illustrates a plurality of frames included in a moving image.

FIG. 4 illustrates an analysis region set for a frame.

FIG. 5 illustrates a frame in a moving image.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

FIG. 1 is a block diagram of an image processing apparatus used for thepresent invention. A personal computer (PC) is used in an imageprocessing method.

A central processing unit (CPU) 101 controls other function blocks ordevices. A bridge unit 102 provides a function for controllingtransmission and reception of data between the CPU 101 and the otherfunction block.

A read-only memory (ROM) 103 is a non-volatile read-only memory andstores a program referred to as a Basic Input/Output System (BIOS). TheBIOS is firstly performed when the image processing apparatus isactivated, and controls basic input/output functions of peripheraldevices including a secondary storage device 105, a display device 107,an input device 109, and an output device 110.

A random access memory (RAM) 104 provides a high-speed readable/writablestorage region.

The secondary storage device 105 is a hard disk drive (HDD) providing astorage region having a great amount of capacity. When the BIOS isperformed, an operating system (OS) stored in the HDD is performed. TheOS provides a basic function that can be used by all applications,management of an application, and a basic graphical user interface(GUI). A user interface (UI) that can realize a function unique to theapplication can be provided by combining the GUIs provided by the OSwith each other.

The above-described OS, implementation programs for other applications,and data used for operations are stored in the ROM 103 or the secondarystorage device 105 as necessary. Further, the image processingapplication performing the processing of the present exemplaryembodiment is stored in the ROM 103 or the secondary storage device 105,and provided as an application activated by a user's operation. Theprocessing described below can be realized when the CPU 101 executes theOS and the programs such as the image processing application stored inthe ROM 103 and the secondary storage device 105 using the RAM 104 as awork memory.

The display control unit 106 performs control for displaying varioustypes of images on the display device 107. For example, the imagesstored in the secondary storage device 105, and the GUI imagesindicating results of the user's operation performed for the OS or theapplication are displayed. A liquid crystal display and a cathode raytube (CRT) display can be used as the display device 107.

An input/output (I/O) control unit 108 provides an interface between aplurality of input devices 109 and output devices 110. A typicalinterface is, for example, a universal serial bus (USB) and a PersonalSystem/2 (PS/2).

An input device 109 is an operation device such as a keyboard and amouse. The I/O control unit 108 inputs a user's instruction to the imageprocessing apparatus via the input device 109.

The I/O control unit 108 connects various types of output devices 110and transmits data thereto. The output device 110 is a printer, forexample. When the I/O control unit 108 outputs print data to the outputdevice, the output device 110 can print the image corresponding to theprint data.

A storage device such as a digital camera, a USB memory, a compact flash(CF) memory, and a Secure Digital (SD) memory card can be connected tothe image processing apparatus. Further, the I/O control unit 108 cantransfer the data such as the image data to the digital camera and thememory, and can also acquire the data from the digital camera and thememory.

According to the present exemplary embodiment, as described below, theframe is extracted from the moving image constituted by a plurality offrames. The data of the moving image is supplied from theabove-described secondary storage device 105, the USB memory, the CFmemory, and the SD memory card connected to the image processingapparatus.

Frame extracting processing according to the present exemplaryembodiment will be described below.

According to the present exemplary embodiment, when the frame includedin the moving image is extracted, an amount of shake and blur isdetermined, and then a frame having a small amount of the shake and bluris extracted as an output target. That is because, in the case of themoving image particularly, brightness and color in continuous frames mayslightly vary, and also a slight movement of a main object, which maycause, affected by compression, the shake and blur. Further, if thecharacteristic of a scene greatly varies, the varied characteristicitself causes the shake and blur. Thus, when the frame is extracted fromthe moving image, the shake and blur in the frame needs to be determinedto extract an appropriate frame as the output target.

According to the present exemplary embodiment, furthermore, when theshake and blur in the frame is determined, analysis results of the shakeand blur in a plurality of regions in the frame is used.

This is because the analysis result in only one region in the frame maynot be able to appropriately evaluate the shake and blur in the frame.This case will be described with reference to FIG. 5.

FIG. 5 illustrates a frame included in the moving image. In FIG. 5, ananalysis region 501 is set near a main object 502 to determine the shakeand blur in the analysis region 501. For example, when the main object502 is moving at a certain speed, if the object is photographed with thecamera as being synchronized with the movement of the object, the objectcan be focused but background may have the shake.

Therefore, even if one analysis region 501 has the small amount of theshake and blur, other regions than the analysis region 501 may have alarge amount thereof, and thus the image may not be preferable as theframe of the output target.

Further, a background region 503, which is not the main object 502 inthe analysis region 501, may have the large amount of the shake asdescribed above. Thus, if the shake and blur in the frame is determinedusing one analysis region 501, the determination result may greatly varydepending on a method for setting the one analysis region.

Particularly, when the moving image is photographed, if the moving imageis photographed by a photographing method in which a region to befocused on is more limited, difference in the amount of the blur betweenthe main object and the background is large. For example, if thephotographing is performed with a diaphragm of a lens opened (with afocus ‘F’ value decreased), a range of a depth of field to be focused onis small. In this case, the background becomes blurred relative to thefocused object. In particular, if the photographing is performed withthe “F” value decreased using a bright lens, the region to be strictlyfocused on is limited also in an object region.

When the analysis region 501 is set for the region to be focused on,even if the main object has the small amount of the shake and blur inthe one analysis region 501, the background may have the large amount ofshake and blur. Further, depending on how much of the background region503 is included in the one analysis region 501, evaluation of the shakeand blur in the frame particularly, greatly varies. In other words, theshake and blur in the entire frame may not be able to be appropriatelyevaluated according to the setting of the one analysis region 501.

Therefore, according to the present exemplary embodiment, a plurality ofanalysis regions are set for one frame and, based on the analysis resultof the plurality of analysis regions, an appropriate frame as the outputtarget is extracted from among a plurality of frames.

FIG. 2 is a flowchart illustrating a frame extracting processingaccording to the present exemplary embodiment. A program correspondingto this flowchart is stored in the ROM 103 or the secondary storagedevice 105. The CPU 101 reads out the program into the RAM 104 andexecutes it, so that the processing illustrated in the flowchart of FIG.2 can be realized.

In step S201, from among the plurality of frames included in the movingimage data, a plurality of frames of an analysis target is selected. Instep S202, for each of the plurality of frames selected in step S201, aplurality of analysis regions is set that is a target of analysisprocessing for determining the amount of the shake and blur. In stepS202, as described above, the plurality of analysis regions are set.Details of the plurality of analysis regions will be described withreference to FIG. 4 below.

In step S203, image quality evaluation is performed in the frame on eachof the plurality of analysis regions set in step S202. As an imageevaluation method, particularly, the amount of the shake and blurincluded in the analysis region is estimated herein. A known techniquemay be used as a method for estimating the amount of the shake and blur.For example, an amount of edge of the analysis region is calculated asan evaluation value of the amount of the shake or that of the blur.

In step S204, the image quality evaluation values in the respectiveanalysis regions corresponding to each other between the peripheralframes are compared with each other. In step S205, based on thecomparison result of the image quality evaluation values of each other,a frame of the output target is finally determined and extracted.

In step S206, the frame of the output target extracted in step S205 isoutputted. For example, the frame extracted from the moving image datamay be displayed on the display device 107 by the display control unit106. Alternatively, print data corresponding to the frame extracted fromthe moving image data may be outputted to the output device 110, whichis a print device, and then the image corresponding to the frame may beprinted on print medium.

Selection of the frame of the analysis target in step S201 describedabove will be described with reference to FIG. 3 herebelow. FIG. 3illustrates a plurality of frames included in the moving image. InstepS201, from among a series of frames included in the moving image, a mainframe 301 is determined. The frame is specified by the user using theinput device 109. Alternatively, the plurality of frames specified foreach predetermined number of frames from among all frames of the movingimage may be set as the respective main frame 301.

The frame near the main frame 301 is set as a peripheral frame 302.Frames continuing to the main frame may be used as a peripheral frame,or the peripheral frames may be set at a predetermined interval in arange having substantially same scene characteristics.

In step S201, the main frame 301 and the peripheral frames 302corresponding to the main frame 301 are selected as the plurality offrames (frame group) of the analysis target on which analysis fordetermining the shake and blur is performed. In step S205, based on theanalysis result of the frames in steps S202 to S204, the frame havingthe small amount of shake and blur is extracted as the frame of theoutput target from among the plurality of frames of the analysis target.

Setting of the analysis region in the frame in step S202 will bedescribed in detail.

FIG. 4 illustrates the analysis region set in the frame. In step S202illustrated in FIG. 2, the plurality of analysis regions illustrated inFIG. 4 is set. According to the example illustrated in FIG. 4, fiveanalysis regions 402 are set for each of the plurality of framesselected in step S201.

According to the present exemplary embodiment, the analysis region isset for the frame included in the moving image having full highdefinition (HD) image quality of 1,920×1,080 pixels, and five analysisregions 402 having a size of 128×128 pixels are set as illustrated inFIG. 4. The size of the analysis region 402 may be set according to thenumber of pixels of the moving image.

In step S203 illustrated in FIG. 2, the analysis for determining theshake and blur is performed for each of five analysis regions in oneframe. Depending on the amount of the shake and blur acquired by theanalysis, the evaluation value indicating the image quality of each ofthe analysis regions is determined. The evaluation value is stored inthe RAM 104, for example. The evaluation value is acquired byquantifying the amount of the shake and blur of the image data in theanalysis region. According to the present exemplary embodiment, theevaluation value is set with a score, which increases in proportion tothe amount of the shake and blur. However, on the other hand, a 100perfect score may be defined for no shake and no blur, and a score ofthe evaluation value may be set to be decreased along with an increasingamount of the shake and blur.

In step S204 illustrated in FIG. 2, each of the evaluation value of thefive analysis regions is compared between frames. At this point, theevaluation values are compared for the respective corresponding analysisregions of each frame. Specifically, two frames are selected from amongthe main frame and the peripheral frames thereof, and the evaluationvalues of the five analysis regions of the respective two framesacquired in step S203 are compared with each other. At this point, theanalysis regions corresponding to each other between the two frames arecompared with each other. For example, the analysis regions disposed inthe center of the two frames are compared with each other. A userfrequently photographs an image such that a main object is arranged inthe center of the frame. Therefore, as described above, by comparing theanalysis regions in the center of the frames with each other, the shakeand blur can be determined for the object common to the two frames.

Further, when the evaluation values of the respective analysis regionsbetween the two frames are compared with each other, the comparison isperformed by a “winning point method”. In the “winning point method”,the evaluation values of certain analysis regions are compared with eachother between the two frames. When difference between both evaluationvalues is a predetermined value or more, the winning points 3 is givento the analysis region in the frame having the higher evaluation value.When the difference between the evaluation values of the two frames isless than the predetermined value, it is considered to be drawn, and thewinning point 1 is given to the analysis regions in the two frames. Thecomparison processing is performed on the five analysis regions. Thewinning points given to the respective five analysis regions in oneframe are added up, and the added winning points are defined as a frameevaluation value of the frame.

The comparison processing between the two frames is performed on theplurality of frames selected in step S201 to be compared with all framesexcept for itself. Each time the comparison processing between the twoframes is performed on the two frames, the frame evaluation value isadded to the respective two frames.

In step S205, based on the frame evaluation value finally added, theframe of the output target is extracted. In step S205, a frame havingthe highest evaluation value is determined to be an optimum frame of theextraction target, and then the optimum frame is determined as theextraction frame. However, a plurality of frames satisfying apredetermined evaluation value may be also determined as the extractionframe.

As described above, according to the present exemplary embodiment, aplurality of analysis regions are set for one frame, and based on theanalysis results of the plurality of analysis regions, the shake andblur of the frame is determined. As described above, when the shake andblur is determined with one analysis region, according to the analysisresult of the one analysis region, the evaluation of the shake and blurof the frame may greatly vary, and thus the image quality of the framemay not be appropriately evaluated.

On the other hand, according to the present exemplary embodiment, sincethe plurality of analysis regions are set, even if the shake and blur ofthe frame cannot be appropriately determined in one analysis region, thedetermination can be appropriately performed in consideration of otheranalysis regions.

As described above, according to the present exemplary embodiment, theperipheral frame 302 is selected with respect to the main frame 301, andthe main frame 301 and the peripheral frame 302 are defined as a framegroup to be analyzed. When the scene is changed in the frame group, theframes corresponding to a plurality of scenes are included in the framegroup. In this case, it is useful to extract the frames having the smallamount of the shake and blur from each of the plurality of scenes,rather than extracting the frame including any of scenes based on theamount of the shake and blur.

Normally, it depends on a photographing scene though, if the frames havethe interval of 0.5 sec or more to less than 1 sec, it is assumed thatthe scene does not greatly vary. Thus, if a range of the peripheralframe is limited to a range of 0.5 sec or more to less than 1 sec,variation of the scene characteristics can be decreased. With thisarrangement, in the frame group which is the analysis target of theshake and blur, the scene can be prevented from being changed. In stepS202, before the shake and blur is determined, the plurality of framesthat are candidates for the determination target of the shake and blurare analyzed, and then the scenes of the plurality of frames may bespecified. When the plurality of frames correspond to a plurality ofscenes, the user may select any of them or, the frame including thescene same as that of the main frame may be selected as the frame whichis the determination target of the shake and blur.

A size of the analysis region illustrated in FIG. 4 will be describedherebelow. As described in FIG. 5, the key factor is that the size ofthe analysis region is set such that the background is not included asmuch as possible. The size of the analysis region may be determinedbased on an assumed photographing scene. Specifically, assuming that amoving child is photographed with a moving image, the size of theanalysis region is set to include a entire body of the child in theframe having the full HD image quality (1,920×1,080 pixels). When theobject is photographed at 80 percent of the 1,080 pixels in a verticaldirection of the frame, and then it is converted into one sixth to onefifth of the 80 percent thereof, which is 144 to 172 pixels. The size ofthe analysis region may be set equal to or smaller than thereof. Asdescribed above, when the size of the analysis region is determinedbased on a ratio of the size of the object relative to the size of theframe, the number of pixels of the analysis region may be determinedaccording to the number of the pixels of the entire frame.

Regarding a method for arranging the plurality of analysis regionsillustrated in FIG. 4, the key factor is that regions not to be analyzedneed to be intentionally, appropriately set. In other words, even if theanalysis regions having a predetermined size are closely arranged overthe entire frame, in the assumed scene as described above, most of thebackground may be flowing background. Therefore, even if the framesincluding the estimated scene are analyzed, accuracy of extracting theframes cannot be improved, and thus the analysis regions need to be setat appropriate portions in the frame.

Since the user tends to desire the frame of the object having the smallshake and blur as the output target, it is desirable that the mainobject be included in the analysis region to be used to evaluate theshake and blur in the frame.

Particularly, when the moving image is photographed, a photographertends to photograph the specific object noticed by himself or herself inthe center of an image. In other words, the main object is oftenincluded in a center portion of the frame, and the specific object isoften included in the center portion thereof in the plurality ofcontinuous frames in the moving image. As illustrated in FIG. 4, theanalysis regions are mainly set at the center portion of the frame.

On the other hand, the specific object noticed by the photographer isnot normally included in an end portion of the frame. When the object ismoving, the scene can be changed at the end portion of the frame duringeven a small lapse of time. In the plurality of continuous frames in themoving image, content is often changed at the end portion of the frame.Thus, as illustrated in FIG. 4, the analysis region is not to be set atthe end portion of the frame.

Assuming that the main object may be photographed in the center of aphotographing composition or near four crossing points (referred to as“golden separation points”) of lines that separate the photographingregion into substantially nine equal parts, the above-described regionsin which the main object is photographed can be arranged as the analysisregion. According to the example illustrated in FIG. 4, as the points atwhich the main object can be photographed, the analysis regions arearranged in the center of the frame and four golden separation points.Alternatively, the analysis region may be arranged around a focal pointof a camera as a center. This is because, as control for photographingwith the camera, if focal control is performed at the focal point, afocus can be adjusted around the focal point.

According to the example illustrated in FIG. 4, five analysis regionsare set, however, the present invention is not limited thereto, and aplurality of analysis regions of six or more or four or less may be set.

Further, as described above in the exemplary embodiment, based on theanalysis result of each of the plurality of analysis regions, the frameof the output target is extracted. However, the analysis result may beweighed depending on the analysis region. At this point, of theplurality of analysis regions, the analysis region arranged at a moreimportant position is weighed. For example, to put more emphasis on theanalysis region arranged in the center of the frame than other analysisregions, when the above-described “winning point method” is performedfor the determination, the winning point given to the analysis regiondisposed in the center of the frame is doubled, and the winning pointgiven to the frame is calculated.

As described above, when the analysis region is set, the size of theregion is set according to the number of the pixels of the moving image.Whether the size setting is appropriate may depend on the content of theimage and photograph information (performance of a lens and settingvalues of a camera at the timing of photographing). A method for settingthe size based on the content of the image and the photographinformation will be described.

According to the description above, as the content of the image, inconsideration of the size of the object with respect to the frame, theanalysis region having the predetermined size is set. The frame isanalyzed to determine the size of the object included therein. Accordingto the size of the object, the size of the analysis region isdetermined. For example, using a known face detecting technique, a faceis detected as the object, and the size of the analysis region isdetermined according to the size of the detected face. At this point,depending on a position of the face in the frame, the position of theanalysis region may be determined. For example, when a plurality offaces are detected in an image, the analysis regions having the sizeaccording to the size of the face may be set at respective positions ofthe plurality of faces.

Further, the photograph information includes a focal (F) valueindicating brightness of the lens and a focal distance. As describedabove, the smaller the “F” value is, the smaller the size of the regionto be focused becomes. Furthermore, when a telescopic lens is used forphotographing, generally, the shake is likely to occur. Therefore, basedon such camera information, the analysis size may be adjusted. Forexample, as described above, when the “F” value is small, the size ofthe analysis region may be set smaller than normal not to include theobjects having the different amounts of blur in the analysis region.

The photograph information corresponding to the above-described movingimage is stored in a header portion of a moving image file when themoving image is photographed. By referring to the above-described headerportion of the moving image file stored in the memory, the “F” value anda type of the lens can be determined.

According to the above description, when the preferable frame isextracted from among the continuous moving image frames, the imagequality between the frames is compared based on the analysis regionscorresponding to each other between the two frames, in other words, theevaluation values of the same locations in the two frames.

However, in the assumed scene as described above, the object may move.According to the present exemplary embodiment, it is assumed that theobject moves while the camera also moves, thereby causing the entirebackground becomes flowing background.

In such an assumed scene, the main object has the small amount of theshake, while the background has a large amount thereof. Therefore, theanalysis region including the main object has the extremely higherevaluation value than those of other analysis regions. In other words,at least one analysis region may be included that has the prominentevaluation value relative to those of other analysis regions. Further,when the object moves and is captured in another analysis region,similarly, the evaluation value of the other analysis region becomesprominent compared to those of other analysis regions.

In this case, of the plurality of analysis regions in the frame, theframe including the analysis region having the particularly highevaluation value may be preferentially extracted. With this extraction,the frame including the moving main object can be extracted. Forexample, distribution of the evaluation values of the plurality ofanalysis regions set for each frame is analyzed, and the analysisregions in the frame are compared to each other to determine whether theframe includes the analysis region having the prominently highevaluation value. Subsequently, the plurality of frames including theanalysis regions satisfying such a standard may be selected from amongthe plurality of frames to determine the frame to be extracted fromamong the plurality of frames based on the evaluation values of theabove-described analysis regions. With this determination, from theframes photographed while the camera is being appropriately movedcorresponding to the movement of the main object (frame having the smallamount of the shake and blur in the main object), the image having thesmall amount of the shake especially in the background can be selected.

According to the above-described exemplary embodiment, the plurality ofanalysis regions are set for one frame, and then the shake and blur ofthe frame is determined based on the analysis results of the pluralityof analysis regions. With this arrangement, even if the shake and blurcannot be appropriately determined in one analysis region, theappropriate determination can be performed in consideration of otheranalysis regions.

Accordingly, for example, when the frame of the output target isextracted from among the plurality of frames included in the movingimage, the frame having the small amount of the shake and blur can beextracted. The output processing of various types can be performed, forexample, the frame extracted from the moving image by such a method canbe displayed, or the frame can be printed by the print device. In such acase also, according to the present exemplary embodiment, since theimage having the small amount of the shake and blur can be extracted,the appropriate image can be extracted to be displayed and printed. Whenthe frame is extracted from the moving image as described above, theplurality of frames extracted from the moving image may be aligned to bedisplayed and printed, or the plurality of frames may be combined to bedisplayed and printed.

According to the exemplary embodiment described above, as the method forevaluating the image quality, the method is described for performing theevaluation based on the determination of the amount of the shake andblur, however in addition, the evaluation method may be performed basedon various analysis results of the image quality such as color as theimage quality, adequateness of contrast, and a feeling of noise. Asdescribed above, various modification can be used as the image qualityevaluation method.

The processing of the exemplary embodiment as described above is notlimited to be performed when the frame is extracted from the movingimage, but can be applied to extract a still image from among theplurality of still images at an interval according to a user'sinstruction. For example, the processing can be used when the imagehaving the small amount of the shake and blur is extracted at theinterval specified by the user so that the user can instantly grasp thegreat amount of the content of the still images archived in a computer.

The image processing apparatus described above in the exemplaryembodiment includes the display device 107 and the output device 110,however, the image processing apparatus may cause an external displaydevice connected to the image processing apparatus to display the imageor an external output device connected to the image processing apparatusto output the image. As an example of the image processing apparatus,various devices may be used, for example, a personal computer (PC), animage pickup apparatus such as a digital still camera and a digitalvideo camera, a print device, and a mobile device such as a cellularphone.

A method is also included in the above-described exemplary embodimentfor storing the program for operating the configuration of theabove-described exemplary embodiment in the non-transitory recordingmedium to realize the functions of the above-described exemplaryembodiment, and for reading the program stored in the non-transitoryrecording medium as a code, and then executing the program in thecomputer.

The non-transitory recording medium is a computer-readablenon-transitory recording medium. Further, the non-transitory recordingmedium in which the above-described program is stored and also theprogram itself are included in the above-described exemplary embodiment.As such a non-transitory recording medium, for example, a flexible disk,a hard disk, an optical disk, an optical magnetic disk, a compact disk(CD)-ROM, a magnetic tape, a non-volatile memory card, and the ROM canbe used.

Further, in addition to the program stored in non-transitory recordingmedium solely performing the processing, the program operating on the OSand performing the operation of the above-described exemplary embodimentin cooperation with functions of other software and an extension boardis also included in the above-described exemplary embodiment.

Furthermore, the present invention can be realized not only when oneprocessor executes the program but also when a plurality of processorscooperate with each other.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2011-161439, filed Jul. 23, 2011, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An apparatus comprising: a memory; and aprocessor coupled to the memory which executes the following: analyzingshake included in each of first regions in a first frame in a pluralityof frames in a moving image, using image data of the first frame andwithout using any image data of another frame; analyzing shake includedin each of second regions in a second frame in the plurality of frames,using image data of the second frame and without using any image data ofanother frame; calculating first evaluation values of the first regionsusing first analysis results of the first regions and without using anyanalysis results of another frame, such that a first evaluation valuefor a first region corresponding to a focus point of a camera among thefirst regions is weighted more than that for other regions among thefirst frame; calculating second evaluation values of the second regions,using second analysis results of the second regions and without usingany analysis result of another frame, such that a second evaluationvalue for a second region corresponding to the focus point of a cameraamong the second regions is weighted more than that for other regionsamong the second frame; comparing, for the first frame and the secondframe, the first evaluation values and the second evaluation values, ina manner such that each of the first evaluation values of the firstregions is individually compared with a second evaluation value of acorresponding region among the second regions, assigning a first pointor a second point to one region among the first regions or thecorresponding region among the second regions, based on the comparisonresult, wherein the first point is assigned to a region having a higherevaluation value out of the one region and the corresponding region, ina case where difference between the first value and the second value isequal to or greater than a predetermined value, and the second pointsmaller than the first point is assigned to both of the one region andthe corresponding region in a case where the difference is smaller thanthe predetermined value, wherein a frame evaluation of each of the firstframe and the second frame is defined based on total points given toeach of the first frame and the second frame, determining an outputtarget frame, from the plurality of frames, based on the frameevaluation value of each of the plurality of frames, and outputting thedetermined output target frame, wherein the comparison between the firstframe and the second frame is performed on the plurality of frames to becompared with all of the plurality of frames except for itself, whereineach time the comparison between the first frame and the second frame isperformed on the first frame and the second frame, the frame evaluationvalue is added to the respective first frame and the respective secondframe, and wherein a frame having the highest frame evaluation valuewhich is finally added, is determined as the output target frame.
 2. Theapparatus according to claim 1, wherein in a case where a first regionincludes a content region corresponding to a content and a backgroundregion corresponding to a background, and a second region includes acontent region corresponding to the content and a background regioncorresponding to the background, analysis results of the content regionsare compared with each other and analysis results of the backgroundregions are compared with each other.
 3. The apparatus according toclaim 1, further comprising determining a size of each of the firstregions in the first frame and a size of each of the second regions inthe second frame based on the first frame and the second frame, and thenexecutes the analysis to each of the first regions of the determinedsize and each of the second regions of the determined size.
 4. Theapparatus according to claim 3, further comprising specifying an objectregion corresponding to an object in the first frame or the secondframe, and determining the size of each of the first regions and thesize of each of the second regions, according to a size of the specifiedobject region.
 5. The apparatus according to claim 3, wherein each ofthe first frame and the second frame is a frame which has beenphotographed by the camera, and the processor determines the size ofeach of the regions according to photograph information corresponding tothe first frame or the second frame.
 6. The apparatus according to claim1, further comprising determining, with respect to each of the firstframe and the second frame, positions of the first regions and thesecond regions such that more regions are arranged at a center portionof each of the first frame and the second frame than at an end portionthereof.
 7. The apparatus according to claim 1, wherein the processorselects the plurality of frames included in the moving image, accordingto an instruction by a user.
 8. The apparatus according to claim 7,wherein the processor selects frames specified for every predeterminednumber of frames, and a frame in the plurality of frames is included inthe selected frames and the other frames in the plurality of frames areselected based on the first image.
 9. The apparatus according to claim1, wherein the processor causes a display device to display the outputtarget frame.
 10. The apparatus according to claim 1, wherein theprocessor causes a printing device to print the output target frame. 11.The apparatus according to claim 1, wherein the processor sets the firstregions in the first frame and the second regions in the second frame,so that the first regions in the first frame are positioned identicallyto the second regions in the second frame, and wherein the processorexecutes the comparison to each of the first regions and each of thesecond regions disposed in the same position in each of the first frameand the second frame.
 12. The apparatus according to claim 1, whereinthe processor determines the output target frame, so that a comparisonresult for a region corresponding to a predetermined position in thefirst frame is more weighed than that for another region.
 13. Theapparatus according to claim 12, wherein the predetermined position is acenter of the first frame.
 14. A method performed by a processor byexecuting a program stored in a memory, the method comprising: analyzingshake included in each of first regions in a first frame in a pluralityof frames in a moving image, using image data of the first frame andwithout using any image data of another frame; analyzing shake includedin each of second regions in a second frame in the plurality of frames,using image data of the second frame and without using any image data ofanother frame; calculating first evaluation values of the first regionsusing first analysis results of the first regions and without using anyanalysis results of another frame, such that a first evaluation valuefor a first region corresponding to a focus point of a camera among thefirst regions is weighted more than that for other regions among thefirst frame; calculating second evaluation values of the second regions,using second analysis results of the second regions and without usingany analysis result of another frame, such that a second evaluationvalue for a second region corresponding to the focus point of a cameraamong the second regions is weighted more than that for other regionsamong the second frame; comparing, for the first frame and the secondframe, the first evaluation values and the second evaluation values, ina manner such that each of the first evaluation values of the firstregions is individually compared with a second evaluation value of acorresponding region among the second regions; assigning a first pointor a second point to one region among the first regions or thecorresponding region among the second regions, based on the comparisonresult, wherein the first point is assigned to a region having a higherevaluation value out of the one region and the corresponding region, ina case where difference between the first value and the second value isequal to or greater than a predetermined value, and the second pointsmaller than the first point is assigned to both of the one region andthe corresponding region in a case where the difference is smaller thanthe predetermined value, wherein a frame evaluation value of each of thefirst frame and the second frame is defined based on total pointsassigned to each of the first frame and the second frame, determining anoutput target frame, from the plurality of frames, based on the frameevaluation value of each of the plurality of frames; and outputting thedetermined output target frame, wherein the comparison between the firstframe and the second frame is performed on the plurality of frames to becompared with all of the plurality of frames except for itself, wherein,each time the comparison between the first frame and the second frame isperformed on the first frame and the second frame, the frame evaluationvalue is added to the respective the first frame and the second frame,wherein a frame having the highest frame evaluation value which isfinally added, is determined as the output target frame.
 15. The methodaccording to claim 14, wherein positions of the first regions and thesecond regions are determined, with respect to each of the first frameand the second frame, such that more regions are arranged at a centerportion of each of the first frame and the second frame than at an endportion thereof.
 16. The method according to claim 14, wherein theplurality of frames included in the moving image are selected, accordingto an instruction by a user.
 17. The method according to claim 14,wherein the first regions of the first frame combined comprise an areathat is less than an area of the first frame and the second regions ofthe second frame combined comprise an area that is less than an area ofthe second frame, and wherein a size of the first regions in the firstframe and the second regions in the second frame are set such that abackground region within the first regions and the second regions areminimized.
 18. A computer-readable non-transitory storage medium storinga program for causing a computer to execute the method according toclaim 14.